<!DOCTYPE html>
<html>
<!-- Created by GNU Texinfo 7.1.1, https://www.gnu.org/software/texinfo/ -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<!-- Copyright © 2006-2023 Free Software Foundation, Inc.

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with the
Invariant Sections being "Funding Free Software", the Front-Cover
texts being (a) (see below), and with the Back-Cover Texts being (b)
(see below).  A copy of the license is included in the section entitled
"GNU Free Documentation License".

(a) The FSF's Front-Cover Text is:

A GNU Manual

(b) The FSF's Back-Cover Text is:

You have freedom to copy and modify this GNU Manual, like GNU
     software.  Copies published by the Free Software Foundation raise
     funds for GNU development. -->
<title>OpenACC Library Interoperability (GNU libgomp)</title>

<meta name="description" content="OpenACC Library Interoperability (GNU libgomp)">
<meta name="keywords" content="OpenACC Library Interoperability (GNU libgomp)">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta name="viewport" content="width=device-width,initial-scale=1">

<link href="index.html" rel="start" title="Top">
<link href="Library-Index.html" rel="index" title="Library Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="index.html" rel="up" title="Top">
<link href="OpenACC-Profiling-Interface.html" rel="next" title="OpenACC Profiling Interface">
<link href="CUDA-Streams-Usage.html" rel="prev" title="CUDA Streams Usage">
<style type="text/css">
<!--
a.copiable-link {visibility: hidden; text-decoration: none; line-height: 0em}
div.center {text-align:center}
div.example {margin-left: 3.2em}
span:hover a.copiable-link {visibility: visible}
-->
</style>


</head>

<body lang="en">
<div class="chapter-level-extent" id="OpenACC-Library-Interoperability">
<div class="nav-panel">
<p>
Next: <a href="OpenACC-Profiling-Interface.html" accesskey="n" rel="next">OpenACC Profiling Interface</a>, Previous: <a href="CUDA-Streams-Usage.html" accesskey="p" rel="prev">CUDA Streams Usage</a>, Up: <a href="index.html" accesskey="u" rel="up">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Library-Index.html" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<h2 class="chapter" id="OpenACC-Library-Interoperability-1"><span>9 OpenACC Library Interoperability<a class="copiable-link" href="#OpenACC-Library-Interoperability-1"> &para;</a></span></h2>

<ul class="mini-toc">
<li><a href="#Introduction-1" accesskey="1">Introduction</a></li>
<li><a href="#First-invocation_003a-NVIDIA-CUBLAS-library-API" accesskey="2">First invocation: NVIDIA CUBLAS library API</a></li>
<li><a href="#First-invocation_003a-OpenACC-library-API" accesskey="3">First invocation: OpenACC library API</a></li>
<li><a href="#OpenACC-library-and-environment-variables" accesskey="4">OpenACC library and environment variables</a></li>
</ul>
<div class="section-level-extent" id="Introduction-1">
<h3 class="section"><span>9.1 Introduction<a class="copiable-link" href="#Introduction-1"> &para;</a></span></h3>

<p>The OpenACC library uses the CUDA Driver API, and may interact with
programs that use the Runtime library directly, or another library
based on the Runtime library, e.g., CUBLAS<a class="footnote" id="DOCF3" href="#FOOT3"><sup>3</sup></a>.
This chapter describes the use cases and what changes are
required in order to use both the OpenACC library and the CUBLAS and Runtime
libraries within a program.
</p>
</div>
<div class="section-level-extent" id="First-invocation_003a-NVIDIA-CUBLAS-library-API">
<h3 class="section"><span>9.2 First invocation: NVIDIA CUBLAS library API<a class="copiable-link" href="#First-invocation_003a-NVIDIA-CUBLAS-library-API"> &para;</a></span></h3>

<p>In this first use case (see below), a function in the CUBLAS library is called
prior to any of the functions in the OpenACC library. More specifically, the
function <code class="code">cublasCreate()</code>.
</p>
<p>When invoked, the function initializes the library and allocates the
hardware resources on the host and the device on behalf of the caller. Once
the initialization and allocation has completed, a handle is returned to the
caller. The OpenACC library also requires initialization and allocation of
hardware resources. Since the CUBLAS library has already allocated the
hardware resources for the device, all that is left to do is to initialize
the OpenACC library and acquire the hardware resources on the host.
</p>
<p>Prior to calling the OpenACC function that initializes the library and
allocate the host hardware resources, you need to acquire the device number
that was allocated during the call to <code class="code">cublasCreate()</code>. The invoking of the
runtime library function <code class="code">cudaGetDevice()</code> accomplishes this. Once
acquired, the device number is passed along with the device type as
parameters to the OpenACC library function <code class="code">acc_set_device_num()</code>.
</p>
<p>Once the call to <code class="code">acc_set_device_num()</code> has completed, the OpenACC
library uses the  context that was created during the call to
<code class="code">cublasCreate()</code>. In other words, both libraries will be sharing the
same context.
</p>
<div class="example smallexample">
<pre class="example-preformatted">    /* Create the handle */
    s = cublasCreate(&amp;h);
    if (s != CUBLAS_STATUS_SUCCESS)
    {
        fprintf(stderr, &quot;cublasCreate failed %d\n&quot;, s);
        exit(EXIT_FAILURE);
    }

    /* Get the device number */
    e = cudaGetDevice(&amp;dev);
    if (e != cudaSuccess)
    {
        fprintf(stderr, &quot;cudaGetDevice failed %d\n&quot;, e);
        exit(EXIT_FAILURE);
    }

    /* Initialize OpenACC library and use device 'dev' */
    acc_set_device_num(dev, acc_device_nvidia);

</pre></div>
<div class="center">Use Case 1
</div>
</div>
<div class="section-level-extent" id="First-invocation_003a-OpenACC-library-API">
<h3 class="section"><span>9.3 First invocation: OpenACC library API<a class="copiable-link" href="#First-invocation_003a-OpenACC-library-API"> &para;</a></span></h3>

<p>In this second use case (see below), a function in the OpenACC library is
called prior to any of the functions in the CUBLAS library. More specifically,
the function <code class="code">acc_set_device_num()</code>.
</p>
<p>In the use case presented here, the function <code class="code">acc_set_device_num()</code>
is used to both initialize the OpenACC library and allocate the hardware
resources on the host and the device. In the call to the function, the
call parameters specify which device to use and what device
type to use, i.e., <code class="code">acc_device_nvidia</code>. It should be noted that this
is but one method to initialize the OpenACC library and allocate the
appropriate hardware resources. Other methods are available through the
use of environment variables and these will be discussed in the next section.
</p>
<p>Once the call to <code class="code">acc_set_device_num()</code> has completed, other OpenACC
functions can be called as seen with multiple calls being made to
<code class="code">acc_copyin()</code>. In addition, calls can be made to functions in the
CUBLAS library. In the use case a call to <code class="code">cublasCreate()</code> is made
subsequent to the calls to <code class="code">acc_copyin()</code>.
As seen in the previous use case, a call to <code class="code">cublasCreate()</code>
initializes the CUBLAS library and allocates the hardware resources on the
host and the device.  However, since the device has already been allocated,
<code class="code">cublasCreate()</code> will only initialize the CUBLAS library and allocate
the appropriate hardware resources on the host. The context that was created
as part of the OpenACC initialization is shared with the CUBLAS library,
similarly to the first use case.
</p>
<div class="example smallexample">
<pre class="example-preformatted">    dev = 0;

    acc_set_device_num(dev, acc_device_nvidia);

    /* Copy the first set to the device */
    d_X = acc_copyin(&amp;h_X[0], N * sizeof (float));
    if (d_X == NULL)
    { 
        fprintf(stderr, &quot;copyin error h_X\n&quot;);
        exit(EXIT_FAILURE);
    }

    /* Copy the second set to the device */
    d_Y = acc_copyin(&amp;h_Y1[0], N * sizeof (float));
    if (d_Y == NULL)
    { 
        fprintf(stderr, &quot;copyin error h_Y1\n&quot;);
        exit(EXIT_FAILURE);
    }

    /* Create the handle */
    s = cublasCreate(&amp;h);
    if (s != CUBLAS_STATUS_SUCCESS)
    {
        fprintf(stderr, &quot;cublasCreate failed %d\n&quot;, s);
        exit(EXIT_FAILURE);
    }

    /* Perform saxpy using CUBLAS library function */
    s = cublasSaxpy(h, N, &amp;alpha, d_X, 1, d_Y, 1);
    if (s != CUBLAS_STATUS_SUCCESS)
    {
        fprintf(stderr, &quot;cublasSaxpy failed %d\n&quot;, s);
        exit(EXIT_FAILURE);
    }

    /* Copy the results from the device */
    acc_memcpy_from_device(&amp;h_Y1[0], d_Y, N * sizeof (float));

</pre></div>
<div class="center">Use Case 2
</div>
</div>
<div class="section-level-extent" id="OpenACC-library-and-environment-variables">
<h3 class="section"><span>9.4 OpenACC library and environment variables<a class="copiable-link" href="#OpenACC-library-and-environment-variables"> &para;</a></span></h3>

<p>There are two environment variables associated with the OpenACC library
that may be used to control the device type and device number:
<code class="env">ACC_DEVICE_TYPE</code> and <code class="env">ACC_DEVICE_NUM</code>, respectively. These two
environment variables can be used as an alternative to calling
<code class="code">acc_set_device_num()</code>. As seen in the second use case, the device
type and device number were specified using <code class="code">acc_set_device_num()</code>.
If however, the aforementioned environment variables were set, then the
call to <code class="code">acc_set_device_num()</code> would not be required.
</p>

<p>The use of the environment variables is only relevant when an OpenACC function
is called prior to a call to <code class="code">cudaCreate()</code>. If <code class="code">cudaCreate()</code>
is called prior to a call to an OpenACC function, then you must call
<code class="code">acc_set_device_num()</code><a class="footnote" id="DOCF4" href="#FOOT4"><sup>4</sup></a>
</p>



</div>
</div>
<div class="footnotes-segment">
<hr>
<h4 class="footnotes-heading">Footnotes</h4>

<h5 class="footnote-body-heading"><a id="FOOT3" href="#DOCF3">(3)</a></h5>
<p>See section 2.26,
&quot;Interactions with the CUDA Driver API&quot; in
&quot;CUDA Runtime API&quot;, Version 5.5, and section 2.27, &quot;VDPAU
Interoperability&quot;, in &quot;CUDA Driver API&quot;, TRM-06703-001, Version 5.5,
for additional information on library interoperability.</p>
<h5 class="footnote-body-heading"><a id="FOOT4" href="#DOCF4">(4)</a></h5>
<p>More complete information
about <code class="env">ACC_DEVICE_TYPE</code> and <code class="env">ACC_DEVICE_NUM</code> can be found in
sections 4.1 and 4.2 of the <a class="uref" href="https://www.openacc.org">OpenACC</a>
Application Programming Interface”, Version 2.6.</p>
</div>
<hr>
<div class="nav-panel">
<p>
Next: <a href="OpenACC-Profiling-Interface.html">OpenACC Profiling Interface</a>, Previous: <a href="CUDA-Streams-Usage.html">CUDA Streams Usage</a>, Up: <a href="index.html">Introduction</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Library-Index.html" title="Index" rel="index">Index</a>]</p>
</div>



</body>
</html>
